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(57)Abstract: 

PROBLEM TO BE SOLVED: To automatically extract a 
keyword related to many constitutional elements from a 
picture to be registered and to register the extracted 
keyword as a keyword for retrieving the picture. 
SOLUTION: A picture input part 1 quantizes an inputted 
picture and generates picture data. An area extracting 
part 2 extracts an area from the picture data. An area 
attribute extracting part 3 analyzes each area and gives 
features related to the color, texture, size, and shape of 
the area as attributes. A picture structure extracting 
part 4 inspects phase relation between respective areas 
and prepares data structure expressing respective areas 
and the phase relation between respective areas. An 
object extracting part 5 scans the data structure while F 
referring to a dictionary and substitutes an object for a 
part or all of the data structure. A keyword extracting |~ 
part 6 extracts the object and the positional relation of L 
the object as the keyword of the objective picture. A 
picture registering part 7 registers the keyword in a 
picture data base relating the keyword to its corresponding picture. 
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* NOTICES * 

JPO and NCI PI are not responsible for any 
damages caused by the use of this translation. 

1 .This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2 **** s hows the word which can not be translated. 
3. In the drawings, any words are not translated. 



CLAIMS 



[Claim(s)] 

[Claim 1] An image input means to quantize the inputted image and to generate image data, A field 
extract means to extract the field which consists of pixels with the description which is similar from said 
image data generated by this image input means, A field attribute extract means to extract as an attribute 
the physical information which the field extracted by this field extract means has, An image structure 
extract means to express due to a phase field with the attribute from which the phase relation between 
the fields extracted by said field extract means was computed, and the image was extracted by said field 
attribute extract means, An object extract means to permute phase-related a part or all between fields 
with the attribute computed by this image structure extract means by the object, A keyword extraction 
means to extract the relative position of an object and an object, and the phase relation between objects 
from the phase relation between fields with the attribute updated with this object extract means as a 
keyword, Image registration equipment characterized by having an image registration means to register 
into a database the keyword extracted by this keyword extraction means with the image data generated 
by said image input means. 

[Claim 2] Said field attribute extract means is image registration equipment according to claim 1 
characterized by making characteristic quantity about the color of one or more fields each, magnitude, 
and a configuration into a field attribute at least using the color of a field, magnitude, a configuration, 
and the characteristic quantity about a texture as a field attribute to extract. 

[Claim 3] Said image structure extract means is image registration equipment according to claim 1 
characterized by extracting the relation between either coincidence, duplication, inclusion, junction or 
exclusion as phase relation between a field and a field. 

[Claim 4] Said object extract means is image registration equipment according to claim 1 characterized 
by judging the object which suits from the attribute of the field which constitutes an object, the phase 
relation between the fields which constitute an object, and the relative relation between the attributes of 
the field which constitutes an object. 

[Claim 5] A field extract means to extract the field which consists of pixels with the description which is 
similar from image data, A field attribute extract means to extract as an attribute the physical 
information which the field extracted by this field extract means has, An image structure extract means 
to express due to a phase field with the attribute from which the phase relation between the fields 
extracted by said field extract means was computed, and the image was extracted by said field attribute 
extract means, An object information storage means to memorize the phase relation between fields, and 
relation with a corresponding object, An object extract means to permute the phase relation between 
fields with said attribute by the corresponding object based on the relation memorized by said object 
information storage means, A phase relation extract means between objects to extract the phase relation 
between said extracted objects, Image registration equipment characterized by having a keyword 
registration means to register the phase relation between the objects extracted by this phase relation 
extract means between objects as a keyword for searching said image data. 

[Claim 6] The step which extracts the field which consists of pixels with the step which quantizes the 
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inputted image and generates image data, and the description which is similar from said generated image 
data, The step which extracts the physical information which the extracted field has as an attribute, and 
the step which computes the phase relation between the extracted fields and expresses an image due to 
said phase field, The step which permutes phase-related a part or all between said fields by the object, 
The step which extracts the relative position of said object and said object, and the phase relation 
between said objects from the phase relation between the fields updated by said object as a keyword, 
The image registration approach characterized by having the step which registers said extracted keyword 
into a database with said generated image data. 



[Translation done.] 
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* NOTICES * 

JPO and NCI PI are not: responsible for any 
damages caused by the use of this translation. 

l.This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2 **** s hows the word which can not be translated. 
3. In the drawings, any words are not translated. 



DETAILED DESCRIPTION 

[Detailed Description of the Invention] 
[0001] 

[Field of the Invention] This invention relates to the image registration equipment and the approach of 
extracting automatically the search key used in order to search the target image from an image, and 
registering it. 
[0002] 

[Description of the Prior Art] As a conventional technique about image retrieval, there is the retrieval 
technique based on the keyword grant in a common database. This determines the language which an 
image registration person gives the keyword of arbitration, or serves as a keyword beforehand, and is the 
technique of giving a suitable keyword with reference to these keyword groups at the time of 
registration. It is the retrieval technique based on the coincidence of a word which chooses the suitable 
vocabulary from the keyword by intention of a retrieval person, or a keyword group, and performs a 
search in the case of retrieval. Although there are easy and an advantage that it can search at a high 
speed if only the configuration of retrieval equipment is easy for this technique and it has registered the 
keyword which a retrieval person means, on the other hand, a registrant needs to give a keyword at the 
time of registration of an image, and it takes time and effort. Moreover, when a registrant differs from a 
retrieval person, a keyword which is different from the difference in both intention may be given, and 
the situation that a retrieval person cannot search the target image occurs. 

[0003] In order to solve the above-mentioned problem, the technique of using an instantiation image as a 
search key instead of a keyword as shown in JP,4-60770,A and JP,6-243178,A is proposed. This 
technique is the technique for realizing retrieval based on the visual description peculiar to an image 
from an are recording image. It is the technique of searching the image which suits conditions to the 
whole are recording image using the form of a structure, magnitude, a location, and the outline image 
that described the color, the texture, etc. further in image information fundamentally. Moreover, in order 
to attain the increase in efficiency of retrieval from an are recording image, the representation color of a 
registered image is mapped, the representation color which the image used as a search key has is 
mapped on this color space, by performing adjustment and evaluation only to the image which exists in 
this near, Oita of extent which is an initial stage is carried out and improvement in the speed is attained. 
However, drawing an image [ needs to create instantiation drawing at every retrieval, and ] searching on 
instantiation drawing also generates a difficult situation. 

[0004] Moreover, in an outline image creation phase as shown in JP,1-130278,A, a plot processor etc. is 
used, the configuration information and its attribute information on a material can be created / registered 
simply, and there is also a method of also carrying out retrieval using the table of configuration 
information and its attribute information. Components here are a desk and an apple, the attribute 
information is a location, magnitude, a posture, etc., and the highly accurate retrieval of it is attained. 
However, it is necessary to add these configuration information / attribute information to an are 
recording image beforehand. 

[0005] Moreover, it considers as the technique of searching a natural image efficiently, for example, 
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there is JP,4-267480,A. This is the technique of component-izing a natural image and using the 
configuration / description information on a component as a search key. However, it is premised on the 
break in of the man by the mouse / keyboard / digitizer on the occasion of component-izing of an image. 

[0006] The conventional image search method explained by the above had required remarkable time and 
effort at the time of registration of an image, or retrieval. As an approach for solving this problem, a 
keyword is extracted from an image automatically or semi-automatically, and there is the approach of 
registering with an image. 

[0007] For example, in JP, 1-73460, A, it is the approach of extracting a feeling word from an image 
automatically, registering with an image, and making this feeling word a keyword. Although the 
retrieval by this feeling word is effective to the specific image group which exists [ pictures ], it is not so 
effective to a common image. It is because feeling changes [ that this is not easy to extract a feeling 
word from a common image, and ] with people. 

[0008] Moreover, there is JP,2- 187864, A as the technique of extracting the keyword about a component 
from an image. This sets up the field of image data and physical information further included to the 
field, such as color information and frequency information, is extracted, and the physical quantity itself 
is made into a keyword, or it makes it possible to connect to language, such as empty and the sea, from 
the physical information on a field, for example. However, there is a limitation in the language tied up 
only from the physical information on a field. 
[0009] 

[Problem(s) to be Solved by the Invention] This invention was made in view of the situation mentioned 
above, extracts the keyword about many components from an image to register automatically, and aims 
at offering the image registration technique of registering this extracted keyword as a keyword for 
retrieval of an image. 
[0010] 

[Means for Solving the Problem] An image input means to quantize the inputted image in image 
registration equipment, and to generate image data in order that this invention may attain the above- 
mentioned purpose, A field extract means to extract the field which consists of pixels with the 
description which is similar from said image data generated by this image input means, A field attribute 
extract means to extract as an attribute the physical information which the field extracted by this field 
extract means has, An image structure extract means to express due to a phase field with the attribute 
from which the phase relation between the fields extracted by said field extract means was computed, 
and the image was extracted by said field attribute extract means, An object extract means to permute 
phase-related a part or all between fields with the attribute computed by this image structure extract 
means by the object, A keyword extraction means to extract the relative position of an object and an 
object, and the phase relation between objects from the phase relation between fields with the attribute 
updated with this object extract means as a keyword, It is characterized by having an image registration 
means to register into are recording equipment the keyword extracted by this keyword extraction means 
with the image data generated by said image input means. 

[001 1] An object points out the expression by the attribute of fields, such as a name of a proper, and "a 
red circle", "a white rectangular head", to things, such as "the component of the image corresponding to 
a keyword, for example, "people's face", and a tree", here. 

[0012] According to this configuration, the relative position of an object and an object and the phase 
relation between objects can be automatically registered as a keyword, can input this at the time of 
retrieval, and can take out the purpose image. 

[0013] Moreover, said field attribute extract means may be made to make a field attribute characteristic 
quantity about the color of one or more fields each, magnitude, and a configuration in this configuration 
at least using the color of a field, magnitude, a configuration, and the characteristic quantity about a 
texture as a field attribute to extract. 

[0014] Moreover, said image structure extract means can extract the relation between either coincidence, 
duplication, inclusion, junction or exclusion as phase relation between a field and a field. 
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[0015] Furthermore, said object extract means can judge the object which suits from the attribute of the 
field which constitutes an object, the phase relation between the fields which constitute an object, and 
the relative relation between the attributes of the field which constitutes an object. 
[0016] Moreover, this invention is set to image registration equipment, in order to attain the above- 
mentioned purpose. A field extract means to extract the field which consists of pixels with the 
description which is similar from image data, A field attribute extract means to extract as an attribute the 
physical information which the field extracted by this field extract means has, An image structure extract 
means to express due to a phase field with the attribute from which the phase relation between the fields 
extracted by said field extract means was computed, and the image was extracted by said field attribute 
extract means, An object information storage means to memorize the phase relation between fields, and 
relation with a corresponding object, An object extract means to permute the phase relation between 
fields with said attribute by the corresponding object based on the relation memorized by said object 
information storage means, A phase relation extract means between objects to extract the phase relation 
between said extracted objects, It is characterized by having a keyword registration means to register the 
phase relation between the objects extracted by this phase relation extract means between objects as a 
keyword for searching said image data. 

[0017] According to this configuration, at least, the phase relation between objects can be automatically 
registered as a keyword, can input this at the time of retrieval, and can take out the purpose image. 
[0018] Moreover, this invention is set to the image registration approach, in order to attain the above- 
mentioned purpose. The step which extracts the field which consists of pixels with the step which 
quantizes the inputted image and generates image data, and the description which is similar from said 
generated image data, The step which extracts the physical information which the extracted field has as 
an attribute, and the step which computes the phase relation between the extracted fields and expresses 
an image due to said phase field, The step which permutes phase-related a part or all between said fields 
by the object, The step which extracts the relative position of said object and said object, and the phase 
relation between said objects from the phase relation between the fields updated by said object as a 
keyword, It is characterized by having the step which registers said extracted keyword into a database 
with said generated image data. 

[0019] Also in this configuration, the relative position of an object and an object and the phase relation 
between objects can be automatically registered as a keyword, can input this at the time of retrieval, and 
can take out the purpose image. 
[0020] 

[Embodiment of the Invention] Hereafter, the example of this invention is explained with reference to a 
drawing. Drawing 1 is the block diagram showing one gestalt of operation of the image registration 
equipment of this invention, the inside of drawing, and 1 — for the field attribute extract section and 4, 
as for the object extract section and 6, the image structure extract section and 5 are [ the image input 
section and 2 / the field extract section and 3 / the keyword extraction section and 7 ] the image 
registration sections. 

[0021] The image input section 1 quantizes the inputted image, and generates image data. The field 
extract section 2 extracts the field constituted from image data generated in the image input section 1 by 
the pixel with the similar description. The field attribute extract section 3 analyzes each field extracted 
in the field extract section 2, and extracts the description about the color of a field, a texture, magnitude, 
and a configuration as an attribute. The image structure extract section 4 investigates the phase relation 
between each field extracted in the field extract section 2, and creates the DS expressing the phase 
relation between a field with the attribute extracted by the field attribute extract section 3, and each field. 
The object extract section 5 scans the DS created in the image structure extract section 4, referring to the 
dictionary which matched the phase relation and the object between attribute conditions and a field, and 
permutes said all all [ a part or ] by the object. The keyword extraction section 6 extracts the physical 
relationship of an object and an object from the DS updated in the object extract section 5 as a keyword 
of an object image. The image registration section 7 connects the keyword extracted in the keyword 
extraction section 6 with a corresponding image, and registers it into an image database. 
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[0022] Drawing 2 is the block diagram showing the example of a configuration which realizes one 
gestalt of operation of the image registration equipment of this invention, the inside of drawing, and 101 
— the image input-process section and 102 — image data storage memory and 103 — the field extract 
processing section and 104 — a field / field attribute storage memory, and 105 — for image structure 
storage memory and 108, as for an object dictionary and 1 10, the object extract processing section and 
109 are [ the field attribute extract processing section and 106 / the image structure extract processing 
section and 107 / the keyword extraction processing section and 111 ] the image registration processing 
sections. 

[0023] In the image input-process section 101, using a scanner etc., the manuscript for an input is 
inputted, it quantizes, and multi-tone RGB color picture data (image data expressed with three elements 
R, G, and B of an RGB system of color representation) are generated. Since the manuscript was set as 
the input target, the scanner was used as an input means here, but when making outdoor scenery 
applicable to an input, you may input with a video camera etc. Or the image drawn using the computer is 
sufficient. The image inputted is once stored in storage, and it can constitute so that it may read, when 
processing. In this example, although RGB color picture data were generated as an input image, this 
invention is not necessarily limited to this, and the image data of a gray scale is sufficient as it, and it 
may use other color coordinate systems, such as L*a*b*. 

[0024] The image data storage memory 102 memorizes the image data generated by the image input- 
process section 101. 

[0025] The field constituted from the field extract processing section 103 by the pixel with the 
descriptions (a color, concentration, a texture, etc.) similar based on the image data memorized by the 
image data storage memory 102 is extracted. Although the technique of extracting many fields by the 
former is proposed, un-hierarchical clustering and the so-called k-average field division technique can 
be used, for example. By this example, based on the hue of each pixel, saturation, and lightness, the k- 
average field division approach is used and the case where all the pixels that constitute an image are 
clustered is shown. Since the field in this invention corresponds to an object called the component in an 
image, for example, a face, an eye, etc., even if an extremely small cluster is obtained, it is almost 
meaningless. Therefore, processing which unifies a cluster may be performed at the end if needed. And 
the field which consists of pixels with the similar description can be extracted by extracting the 
contiguity pixel belonging to the same cluster by labeling. Here, it is not necessary to memorize all the 
fields extracted by labeling, and a threshold may be established to area size, a total, etc. and the field to 
memorize may be restricted. 

[0026] Although k-average field division and labeling performed field extract processing in this 
example, if this invention is the technique of the ability to extract the field constituted by the pixel with 
the description which does not limit to this and is similar, it is usable by any technique. There is a 
method of extracting as a field the closed region which extracts the pixel equivalent to an edge from an 
image as other field extract technique, and is constituted by this pixel etc. 

[0027] A field / field attribute storage memory 104 memorizes the field extracted by the field extract 
processing section 103 as field data. Although the configuration of field data can also consider various 
approaches, a labeling image as shown in drawing 3 constitutes field data from this example, and it 
memorizes in a field / field attribute storage memory 104. Among drawing, it corresponds and the 
numeric value in a mass is the number (a field number is called henceforth) of a proper to the field to 
which the corresponding pixel whose one mass is one pixel belongs. The configuration approach of the 
field data in this invention is not necessarily restricted to a labeling image, gives the unique number to 
each pixel in an image, and may constitute field data using other approaches, such as the approach of 
expressing with the set of the unique number given to the pixel which belongs each field to the field, and 
the approach of expressing with the profile point-set of a field. 

[0028] In the field attribute extract processing section 105, the attribute corresponding to each field 
memorized by a field / field attribute storage memory 104 is extracted. The extracted attribute is related 
with a corresponding field and memorized in a field / field attribute storage memory 104. The attribute 
in this invention can express the property about the color of a field, magnitude, a configuration, and a 
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texture, and characteristic quantity, such as an average hue, the histogram of saturation, area, a 
parameter showing maximum length and an approximation configuration, a degree of roundness 
(needlelike), average edge reinforcement, and spatial frequency, can be used absolutely. This inventions 
do not limit the attribute to extract to the above-mentioned characteristic quantity, and if they are the 
color of a field, magnitude, a configuration, and the characteristic quantity showing the property about a 
texture, they are good anything. However, it is desirable that one uses each the characteristic quantity 
showing the property about the color of a field, magnitude, and a configuration at least. In this example, 
in order to simplify, it uses as an attribute which extracts the average hue of a field except the 
description about a texture, average saturation, average lightness, the number of configuration pixels, a 
needlelike degree, and the direction of a main shaft. It is memorizable in a field / field attribute storage 
memory 104 as a table (an attribute table is called henceforth) made into the field number and the group 
as these descriptions were calculated from field data by the known approach, for example, it was shown 
in drawing 4 . 

[0029] Moreover, the description which is equivalent to the attribute of a field in the process of this field 
extract depending on the field extract approach used in the field extract processing section 103 is 
computable. In such a case, when it computes, this description can be related with the field which 
corresponds as an attribute, and it can also memorize in a field / field attribute storage memory 104. 
Since the average hue of each field, average saturation, and average lightness can compute the number 
of configuration pixels of each field in the case of labeling in the case of k-average field division, he is 
trying to memorize these attributes in a field / field attribute storage memory 104 in field extract 
processing in this example. Moreover, although it is computable from field data, all the attributes (a 
needle-like degree, the direction of a main shaft) computed here can also be constituted so that an 
attribute may be computed with reference to the image data memorized by image data storage memory if 
needed (for example, the description about a texture) depending on the attribute to be used. 
[0030] The image structure extract processing section 106 extracts the phase relation between each field 
using the field data memorized by a field / field attribute storage memory 104. The phase relation 
between the fields in this invention points out the relation between fields, such as coincidence / 
inclusion / duplication / junction / exclusion, and a field, and it can have the distance and the direction of 
[ between fields ] as detailed relation. In this example, since field division extracted each field, two 
fields of arbitration do not overlap, therefore, the phase relation of two fields — "— it contains ~ it 
is /contained (inclusion) — " — it becomes either "adjoining (junction)" of "being separated (exclusion)." 
The phase relation between this field is computable by scanning the field data (labeling image) 
memorized by a field / field attribute storage memory 104 to a horizontal direction and a perpendicular 
direction, and extracting the field number which adjoins for every field. However, to use this approach, 
it is necessary to create the field which expresses the outside of an image with the rim part of an image 
as shown in drawing 5 . The pixel by which labeling was carried out for the numeric value of "0" in 
drawing is a field showing the outside of an image. As a result of extracting the field contiguous to each 
field, it comes to be shown in the term of the adjoining field list of number of drawing 6 . This result 
shows that the field of the field numbers 2 and 3 where only the one number of adjoining fields is is 
included to the field of that field number 1 that adjoins. And the list of fields which is not included and 
has been joined by removing the field numbers 2 and 3 can be obtained from the adjoining field list of 
number of the field number 1. Moreover, though natural, the field (at this example, a field number is the 
field of 0) which expresses the outside of an image from the list of fields in which final phase relation is 
shown must be removed. The extract approach of the phase relation to this invention is not limited to the 
above-mentioned approach. Moreover, in this example, although adjacency was computed about four, 
you may compute about eight. The distance and the direction of [ between the fields used as detailed 
relation between fields ] can be acquired by computing the distance and the direction on the basis of a 
coordinate of a center of gravity of a field. 

[003 1] The image structure storage memory 107 memorizes the phase relation between the fields created 
in the image structure extract processing section 106. In this example, the graph structure (image 
structure graph is called henceforth) which has a field number as shown in drawing 7 in a node, and has 
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the phase relation between fields in an edge is created and memorized. These values can also be given to 
an edge when the distance and the direction of [ between each field ] are being computed as detailed 
relation between fields. The field numbers from 1 to 1 1 which the image structure graph shown in 
drawing 7 is what was created from the labeling image (one closed region expresses one field) shown in 
drawing 8 , and were shown in drawing 7 shall have pointed out the field of the face, a left eye, a right 
eye, opening, left eyebrows, right eyebrows, the hair of hair, a left ear, a right ear, a neck, and a 
background in order. Although the image structure graph in this example described only the relation 
between the fields which include or adjoin, when a field which allows duplication is extracted in the 
field extract processing section 103, the phase relation between fields, such as coincidence and 
duplication, can be added. 

[0032] In the object extract processing section 108, an object is extracted from the image structure graph 
memorized by the image structure storage memory 107, referring to the object dictionary 109. The 
object in this invention points out the expression by the attribute of fields, such as a name of a proper, 
and "a red circle", "a white rectangular head 11 , to things, such as "people's face" and a "tree." 
[0033] The object dictionary 109 holds two or more groups (an object item is called henceforth) of the 
graph structure (an object graph is called henceforth) and the object conditions which described the 
name of an object, the attribute of the field which constitutes an object, and the phase relation between 
fields. The object graph memorized in the object dictionary 109 is the same format as image structure 
graph. A different point is a point with the attribute conditions corresponding to the number which each 
node of an object graph shows instead of an attribute. Henceforth, the number which the node of an 
object graph has is called an attribute condition number. This attribute condition is conditions for 
investigating whether the corresponding node of image structure graph and an object graph suits. 
Moreover, with object conditions, it is a regulation about the relative relation between the attributes of 
the field which constitutes an object, for example, those of Field A is conditions, such as more than 
twice of the number of configuration pixels of the number field B of configuration pixels, and 3 or less 
times. The example of the attribute conditions of the part corresponding to the object item corresponding 
to the object "people's face", and this object item is shown in drawing 9 . Among drawing, the face, and 
302 and 303 are equivalent to an eye, and 304 is equivalent to the hair of hair for the attribute condition 
number 301. 

[0034] Below, it explains, referring to the flow chart which shows an example of the object extract 
approach to drawing 10 . First, the attribute table memorized by a field / field attribute storage memory 
104 is sorted in descending order with the number of configuration pixels (S201). This is based on the 
rule of thumb that possibility that the fields where area is bigger are main components is high. 
Subsequent processings make a processing object the field (field number) of the sorted attribute table at 
order (S202). The attribute conditions registered into the object dictionary 109 are searched, and the 
attribute of the field of a processing object creates the list of attribute condition numbers which fulfills 
conditions (S203). An object graph and image structure graph with the attribute condition number of this 
list are compared (S204). This comparison investigates whether after making in agreement the node of 
image structure graph with the field number of a processing object, and the node of an object graph with 
the attribute condition number extracted by S203, the phase relation corresponding to all the attribute 
conditions and edges corresponding to the node which this object graph has suits all the image all [ a 
part or ]. When this suits, further, the object conditions corresponding to this attribute condition number 
are applied, and it asks for whether finally it suits or it does not carry out (S205). S208 is processed 
when it is judged that it finally does not suit. When suited, the subgraph with which image structure 
graph suited is permuted by the corresponding object (S206). This permutation can be based on a node 
with the number which can determine an object as a meaning, and the number (an object number is 
called) of the corresponding object item registered into the object dictionary 109 can be used. Correction 
of the phase relation corresponding to deletion of an edge or an edge, the distance between fields, and a 
direction is also made with this permutation. The center of gravity of the object for computing distance 
and a direction is computable from the field which unified the field included in the subgraph of the 
image structure graph permuted by the object. And the attribute corresponding to the field number and 
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this which are contained in the subgraph of the image structure graph permuted by the object is deleted 
from the attribute table sorted by S201 (S207). In S208, if it investigated how [ that processed all fields ] 
it was and the unsettled field remains, it will return to S202 and processing will be repeated. 
[0035] As a result of the object extract processing section 108, some subgraphs contained in image 
structure graph are permuted by the object, and a graph which has an object number or a field number in 
each node of image structure graph is obtained. Object extract processing is performed to the image 
structure graph shown in drawing 7 , and the graph obtained as a result of suiting the object graph 
registered into the object dictionary 109 which this showed to drawing 9 is shown in drawing 1 1 . The 
field numbers 1, 2, 3, and 7 of drawing 7 shall suit the attribute condition number 301,302,303,304 of 
drawing 9 , respectively, and the object conditions shown in drawing 9 shall also be fulfilled. The edge 
between each node (phase relation) conforms, and it permutes by the node in which the subgraph which 
consists of field numbers 1, 2, 3, and 7 of drawing 7 has the object number 100 corresponding to the 
object "people's face." 

[0036] Here, although the example which makes the name of the proper of a thing an object was shown, 
objects, such as "a red circle" and "a white rectangular head", can be extracted by creating an object item 
which makes the name of an object the expression about the attribute of fields, such as "a red circle" and 
"a white rectangular head", at the object dictionary 109. 

[0037] Moreover, in this example, although the comparison of an object graph and image structure 
graph extracted the object by whether each node and an edge suit or it does not carry out, it takes in the 
concept of a goodness of fit here, and can permute it by the object which corresponds the subgraph of 
image structure graph with the highest goodness of fit. For example, it is realizable by constituting from 
a function which returns the goodness of fit, for example, a fuzzy function, by giving an attribute for the 
attribute conditions currently held in the object dictionary 109. 

[0038] It is necessary to draw up the object dictionary 109 in this invention beforehand. Image structure 
graph which has been mentioned above to the image of two or more men's face as an example in this 
creation approach is created, and two or more image structure graph is obtained. And the object graph 
corresponding to the object "people's face" can be created from the phase relation of the edge which is 
common for the attribute of the node (field) which corresponds with two or more image structure graph. 
Moreover, relation it can consider that is common among the relation between the attributes which the 
node of two or more obtained image structure graph has can be made into object conditions. 
[0039] The keyword extraction processing section 110 analyzes the image structure graph with which 
the subgraph was permuted by the object by the object extract processing section 108, and extracts the 
relation between objects as a keyword. In this example, the keyword "people's face is the center of an 
image" can be extracted as the relative location in the image of the object (name peculiar to a thing) 
which becomes settled from the object number which exists in image structure graph, and an object, for 
example, a location of the object "people's face", and this object **. Moreover, the relative physical 
relationship between the extracted objects, for example, the keyword of "being a signal at the upper left 
of an automobile", can be extracted as relation between objects. 

[0040] The image registration processing section 1 1 1 relates with an input image the keyword extracted 
in the keyword extraction processing section 110, and registers it into are recording equipment (not 
shown). 

[0041] As mentioned above, by using the explained image registration equipment, the time and effort 
which gives a keyword by the help at the time of registration of an image can be saved, and easy and the 
retrieval equipment with which the target image is searched at a high speed can consist of specifying a 
keyword. However, it is more desirable to constitute image retrieval equipment so that the synonym of 
the keyword specified as a search key at the time of retrieval may also be searched in addition to a 
search key, since the keyword which is extracted by the image registration equipment by this invention, 
and is registered with an image is mainly a name peculiar to a thing. 
[0042] 

[Effect of the Invention] As mentioned above, since according to this invention the keyword which has 
objectivity automatically is extracted from an image and it can register with a database with an image 
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only by inputting an image registering as explained, when a registrant registers an image, the time and 
effort which gives a keyword can be saved. Moreover, since it carries out based on the keyword 
registered with the image, directions of a search key are easy and, as for retrieval, can also simplify the 
configuration of retrieval equipment. 
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* NOTICES * 

JPO and NCI PI are not responsible for any 
damages caused by the use of this translation. 

1. This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2. **** shows the word which can not be translated. 
3. In the drawings, any words are not translated. 



DESCRIPTION OF DRAWINGS 



[Brief Description of the Drawings] 

[Drawing 1] It is the block diagram showing the image registration equipment of the example of this 
invention as a whole. 

[Drawing 2] It is the block diagram showing the example of a configuration of the image registration 
equipment of an example. 

[Drawing 3] It is drawing showing the example of the labeling image expressing the field data in an 
example. 

[Drawing 4] It is drawing showing the example of the attribute table of an example. 

[Drawing 5] It is drawing showing the example of the labeling image used in case the phase relation 

between fields is extracted in an example. 

[Drawing 6] It is drawing showing the example which extracted the adjoining field for every field in the 
example. 

[Drawing 7] It is drawing showing the example of the image structure graph of an example. 
[Drawing 8] It is drawing showing the example of the labeling image corresponding to the image 
structure graph of drawing 7 . 

[Drawing 9] It is drawing showing the example of the object dictionary of an example. 

[Drawing 10] It is drawing showing the example of the processing flow of the object extract art of an 

example. 

[Drawing 11] It is drawing showing the example of the image structure graph after object extract 
processing of an example. 
[Description of Notations] 

1 Image Input Section 

2 Field Extract Section 

3 Field Attribute Extract Section 

4 Image Structure Extract Section 

5 Object Extract Section 

6 Keyword Extraction Section 

7 Image Registration Section 

101 Image Input-Process Section 

102 Image Data Storage Memory 

103 Field Extract Processing Section 

104 Field / Field Attribute Storage Memory 

105 Field Attribute Extract Processing Section 

106 Image Structure Extract Processing Section 

107 Image Structure Storage Memory 

108 Object Extract Processing Section 

1 09 Object Dictionary 

110 Keyword Extraction Processing Section 
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111 Image Registration Processing Section 
[Translation done.] 
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S. ^o^ll-eti, 7^1" J; 9 ft, K 

T^5) ri>£fNcLfcfc<£>-e, [3 7[£^L*:ia>bl 1 
*-CO*«»*tt, ll^ N Kffi, *B, n> 

**** »*>^ fes, *:n\ ^(nmmzm 

tf, MfcSrtM- J: 5 ttMtt*MMU*Hff l 0 3 T-» 

[0 0 3 2] t/v'x^ httaSM^SPl 0 
V 1 0 7^15lS^nfciij^«3t>/7 7^^^-yv ; ni^ h 

i". 

[0 0 3 3] 0 9 ft. *~f*s*9 b 

B©ffi*BBB«*rB*Lfc^5 7*JBt *-f*?*9 

icmt^iMft^oti/^^tfe^o gut* 

h^77^ <D*ffSiT 5 y - K#B**-*a»if p j^StB'* 

i-*«BJ-e*>9, BttAaajftj&BBttfltttB 

5, it?*?*.* hSli:^t^x^ h^BI-awS-*- 
ltfBffi, 3 0 2<!: 3 0 3^g t 3 0 4^g(D^|:tta 

[0034] BTttt, btArnxmo—mi: 

mi 0l:/Tt7D-ft- H*:#IHLtt3ftSblftWi-S d 
5Att^-^/uSr**W*»-e#)Hlcy-h-r5 (S2 



^0 

oi) D :ftit, Bfllo*#4cii«is^««fliAK* 
td«Last*t-t-* (s.2 o 2) e ^-y^oL^ hg?#i o 

/SlTS (S 2 0 3) 0 ^ h(7)JStt^{4=#^-^^0 

*-f*?*9 b??7 kmmtfe?yyk*vcm'Z> (s 
2 0 4) 0 r<oit»ti, fi9&*9M»3-frttoB1fc 

io f85i^ 7(7)7- Ki S 2 0 3T-ttffl^tLfcJRtt^frS 

T\ Z.<D*Zfi/^t b?77<oft<>;-Y\z.$t$stz>-t 
*<x<oWk&1kftk=. v v^MJS-T fiffiffiBBfll^BMHfHt 
^ 9 7 <7>-g!5 * fc tt-t-^ T {djl-a-r 5 ^ if p A'frM't 

aHSi-**-^** h*fl=4:BfflL-(\ ft^lcig^ 
*a»L*V^£#ifr* (S 2 0 5) 0 *»WK»<&Lft 
V^*JBf$nfc»^fi, S 2 0 8«I^3o B 
^LfcJi^te, Bflft«iS^7 7<z>B^Lfc»^^77Sr 

20 Sistst^^x^ sr*atfti-s (S206) o *(ds 

mtisrzr^^^ b£-mz&:7£-rZ>Zk(DX$Z^$: 
Si 0 9^c^^t^rv^5^1-5^y r v ; ^^ 

yv>^* htciatftStLfcBttWiS^v^^SB^^^^Jc 
4Stit^5a«S:«^ Lfcfi«A»5>|[ttlt5 ^ k &X 

^-T^M^^S 2 0 1 T?y- h SixfcJRtt^-^/u^b 
gij^a (S 2 0 7) „ S2 0-8ffit^t«)«8l:o 

ot^jitfs 2 o 2 {zm-ixmm&m*) &-f- 0 
[oo35] irzf it* ^ bt&m&m^ i o 8 (omsk. m 
mmm^yy\c^tix^^^<^(o^^yy^Jt 

h»Sl 0 9i:S»lTfc§^i?x^ 

»i, 2, 3, 7d5-tn^nH9want*ft#^3 o 

1, 3 0 2, 3 0 3, 3 0 4ICMU @9l^ttt 

(oocyv? mnmm ^B*u-ca3 9 % B7om*» 

1, 2, 3, 7frbt£Z>Ufti?7 7& TA^MJ <b^9 
so y- Kfcfi»Slxrv>5c 
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[0 0 3 6] i>(0<om : fi<o%ifr$:jr7 r i?*? 
.[0 0 3 7 ] ifc, r<O0i|T*fi, t/^x^ Y^fyyt 

[0 0 3 8] *«WtC*5JtS^^^ hgfSl 0 9(3 

[0 0 3 9] *-!7-Kil6ffl*3HBl 1 On, t^yx 
* httffl&afflU 0 8 J: D »»^57#^x^ h 

<£>Sj ^p^v^? h £ -o^y^^ b<D<D&.W 

t^x^ h©n<o«*ttt4tt«H«, ran, 

[0 0 4 0] ■ffeXftft9A 1 l 1 tt, ?7— KttW 

[0 0 4 1 ] Kg] Lfcli«X®^H4r^^5 r £ 
T% ®«OXS^icA^(-J:f9^r--!7-K^W-^-r5^ 

5 0 nu ^RKic.taBifcsaasBicioTttfflsn 

[0 0 4 2] 



(7) 

12 

If, »*Lfcl*B«S:Aai-5ratT\ SSittKSIRtt 
OfcS*— 57-KS:Bflk^bi*fflLr, fllft t £ t 
-^-c-^lilftT'tSfc*, «»#3ftSB«Sr»aki-S 
B*fc*-!7— K*tt4-"*-5#B«:*< r fc^-e#5 d * 

0*«t> 6. 
[BB©«#/j:ttW] 
io [mi] *»M©*l£fi»J^H«»S3gKS:^ft:t LT 
^i"^n y *B"C*>«. 

[ID 2 ] IWj?)I^Iigt<7)MMIr^t/p y 

[B3] mmm & «■ s«f - * «r«m-r * 9 ^ y 

>^B<ft^0iJS:^-ria-C*>So 

[in 4 ] ^j£^j^jH4x-y/Ko«^^-rnr*fc5 0 

[ID 5 ] **«Jc*3V^r««B^fl[tBH«Srttaf S 

m^m^zr^v >"^B«o«s:*-rB-e*s. 
[id 6 ] sat«^*5v^r««r t <o»«««sram u 

[ID 7 ] IBfiW^BIMPlii^ 70«t*tB-C* 
[ID 8 ] ID 7 «M1^7 7lc^j£f5 SI >^ 
[ID 9] **««>;*-^:»l* h#»o0iJ«:a*1-H-C*> 

So 

[010] SBtB^^^^i * httttJMS^rffio^S 

[Hi 1] jBMyoa-^i* h*tti^g^if«* 
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